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Researcher  Role  in  Aviation  Operations 

Hans  Hoermann 
German  Aerospace  Center 
Pamela  S.  Tsang 
Wright  State  University 
Michael  A.  Vidulich 
Air  Force  Research  Laboratory 
Amy  L.  Alexander 
MIT  Lincoln  Laboratory 

This  chapter  is  based  on  the  Researcher  Plenary  Panel  held  at  the  International 
Symposium  on  Aviation  Psychology  (ISAP)  in  May  of  2015.  This  Panel  followed  the 
Practitioner  Plenary  Panel  held  on  the  preceding  day  of  the  symposium.  The  overarching  goal  of 
the  two  sessions  was  to  foster  a  dialogue  between  operational  personnel  and  researchers  towards 
a  safer  and  more  efficient  sky.  The  charge  to  the  practitioner  panelists  was  to  inform  the  aviation 
community  of  their  operational  challenges.  Their  thoughts  and  discussions  are  captured  in 
Chapter  2  of  this  volume.  The  charge  to  the  researcher  panelists  was  to  explore  best  approaches 
that  would  bridge  the  gaps  between  basic  research  and  current  practical  applications.  The  value 
of  use-inspired  basic  research  was  discussed  to  a  great  extent  by  Stokes  (1997).  That  use-inspired 
research  would  be  a  good  path  towards  accelerating  the  process  of  putting  basic  knowledge  to 
practical  use  will  be  revisited  in  this  chapter. 

The  chapter  begins  with  briefly  presenting  the  missions  of  an  example  research  and 
development  organization  to  offer  a  glimpse  of  its  research  operations.  Research  approaches  and 
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practice  toward  a  safer  and  more  efficient  sky  will  then  be  discussed.  Since  collaborations 
between  researchers  and  practitioners  are  critical  for  the  success  of  use-inspired  research,  means 
to  facilitate  their  collaborations  are  explored.  Finally,  a  call  for  action  moving  forward  will  be 
presented. 

Missions  of  Research  and  Development  Centers 

To  provide  a  glimpse  of  the  natural  habitat  of  researchers,  the  organization  and  missions 
of  the  German  Aerospace  Center  (DLR)  will  be  briefly  described.  The  German  Aerospace  Center 
is  a  large  R&D  organization  with  many  subunits.  It  has  the  mandate  to  respond  to  high  level 
guidance  and  aviation  psychology  research  is  incorporated  into  an  extensive  multi-faceted 
portfolio  that  includes  many  scientific  and  technological  domains. 

German  Aerospace  Center  (DRL) 

Research  institutions  in  aeronautics  or  astronautics,  such  as  the  German  Aerospace 
Center  (DLR)  or  the  National  Aeronautics  and  Space  Administration  (NASA)  have  to  position 
themselves  as  to  the  extent  to  which  they  are  committed  to  basic  and  applied  research.  This  will 
be  decided  strategically  by  the  stakeholders  with  regard  to  the  availability  of  research  facilities 
and  workforce  capabilities.  Due  to  the  culturally  and  politically  diverse  conditions  in  Europe  and 
the  European  Union,  harmonization,  interoperability,  and  mutual  alignment  have  always 
challenged  the  definition  of  any  large-scale  research  agenda  such  as  DLR. 

The  DLR  has  roots  back  to  the  “Aerodynamic  Research  Establishment”  which  was 
founded  in  1907  in  Goettingen.  In  the  1960s  and  70s  it  merged  with  several  other  aeronautical 
research  institutions  and  became  in  1969  the  German  national  research  institution  for  aeronautics 
and  astronautics  with  about  8,000  employees  and  its  headquarters  are  in  Cologne.  The  DLR  sees 
its  general  mission  in  addressing  societal  questions  on  behalf  of  public  customers  by  conducting 
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research  that  enhances  global  mobility  and  safety  while  preserving  environmental  resources.  It 
has  committed  itself  to  bridge  the  gap  between  basic  research  and  innovative  applications  and  to 
transfer  knowledge  and  research  results  to  the  industry  and  the  political  sphere  through 
mediation  and  consultation  as  well  as  through  the  provision  of  services.  Specific  enablers  for 
these  strategic  objectives  are  33  discipline  oriented  institutes  and  contractually  regulated  national 
and  international  partnerships  with  universities,  industry,  other  research  organizations  and  the 
public. 

Air  traffic  has  recorded  substantial  growth  worldwide  over  the  past  decade,  which,  in  all 
likelihood,  will  continue.  Such  growth  cannot  be  sustained  without  consequences  for  the 
standards  and  requirements  that  the  air  transport  infrastructure  must  meet.  For  example, 
questions  arise  regarding  the  impact  on  the  environment  and  climate  with  ever-increasing 
urgency.  Mobility,  communication,  climate  change,  demographic  development,  shortage  of 
resources,  safety  and  security  are  among  the  grand  challenges  of  today.  How  can  research  in 
aeronautics  help  to  enable  sustained  mobility  in  a  demand-oriented,  future-proof  and 
environment-friendly  manner? 

Key  features  of  DLR’s  research  agenda  lie  in  the  holistic  consideration  of  the  air 
transport  system  as  essential  for  achieving  future  objectives.  International,  and  multidisciplinary 
collaboration  is  an  indispensable  condition  to  treat  these  complex  questions  with  greater 
effectiveness.  For  example,  a  long-term  cooperation  agreement  with  NASA  has  recently  been 
renewed.  With  respect  to  aviation  this  agreement  currently  includes  collaborative  efforts  in 
aerodynamics,  air  traffic  management  (ATM),  and  climate  research. 

The  basic  structure  of  DLR’s  aeronautics  program  was  established  in  2007  to  align  with 
the  European  Vision  2020  (European  Commission,  Group  of  Personalities,  2001)  and  the 
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corresponding  strategic  research  agenda.  DLR’s  main  aims  are:  (a)  to  increase  the  efficiency  of 
the  air  transportation  system;  (b)  to  increase  the  cost-effectiveness  of  development  and  operation; 
(c)  to  reduce  aircraft  noise  and  harmful  emissions;  (d)  to  increase  the  quality  of  air  transportation 
for  passengers;  and  (e)  to  increase  safety  in  the  face  of  growth  and  external  danger. 

In  this  research  program,  the  applied  industry  relevance  should  also  be  associated  with 
high  scientific  aspirations.  To  this  end,  the  program  promotes  specific  application-oriented 
projects.  However,  basic  research  also  maintains  an  appropriate  status.  Among  the  currently 
promoted  research  topics  are  listed  below: 

•  Extended  capabilities  to  analyze  and  evaluate  the  overall  air  transportation  system,  which 
includes  air  traffic,  airports  and  flight  guidance,  taking  weather  and  environmental 
aspects  into  account.  This  aims  at  the  performance  optimization  of  the  entire  air  traffic 
management  (ATM)  system  including  its  environmental  compatibility. 

•  Development  of  simulation  procedures  to  support  design,  evaluation  and  certification 
programs  in  aeronautics.  This  aims,  for  example,  at  the  expanded  range  of  helicopters  to 
all  weather  conditions. 

•  Further  development  of  experimental  techniques,  equipment  and  systems  for  validating 
technologies  and  simulation  tools  in  ground  and  flight  tests.  The  research  priorities 
include  for  example  detailed  investigations  of  fossil-based  and  alternative  fuels, 
reliability,  ignition  and  combustion  stability  in  commercial  jet-engines. 

•  Work  for  the  human-machine  interaction  in  the  areas  of  the  cockpit,  cabin  and  air  traffic 
control  work  areas,  taking  psychological  and  medical  aspects  into  account.  This  includes 
human  factors  research  on  adaptive  pilot  and  controller  assistance,  displays  and  sensors 
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for  improved  handling  qualities  of  the  aircraft,  increased  safety,  reduced  workload  and 
enhanced  situation  awareness  during  the  entire  operation. 

•  Research  work  for  a  better  understanding  of  the  climate  impact  of  air  transportation  and 
especially  for  suitable  emission  reduction  measures  in  all  areas  of  the  air  transportation 
system. 

Transcending  across  all  these  fields,  the  research  work  is  supported  by  a  broad  range  of 
large  scale  facilities  -  such  as  wind  tunnels,  research  aircraft,  cockpit  and  tower  simulators, 
combustion  chambers  and  several  test  facilities  for  turbines,  structures  and  materials  -  as  well  as 
the  necessary  infrastructure  for  demanding  numerical  simulations.  These  facilities  provide 
excellent  conditions  to  pursue  research  programs  in  line  with  international,  primarily  European, 
strategies,  objectives  and  funding  measures  such  as  European  Vision  2020  and  Flightpath  2050 
(European  Commission,  High  Level  Group  on  Aviation  Research,  2011). 

Research  and  Practice  towards  a  Safer  and  More  Efficient  Sky 
Generation  versus  Exploitation  of  Knowledge 

For  many  people,  including  practitioners  and  students,  the  word  science  is  often 
associated  with  hard  study,  complicated  mathematical  equations,  voluminous  textbooks  and  busy 
scientists  wearing  white  lab  coats  while  talking  among  themselves.  While  such  impressions  may 
be  based  on  individual  empirical  experiences  they  reflect  only  a  few  aspects  of  science,  which 
leans  towards  the  acquisition  of  new  knowledge  by  basic,  laboratory  research  efforts.  In  the  long 
run,  science  is  driven  by  societal  and  market  needs  that  have  been  identified  and  prioritized  by 
policymakers,  industries,  and  research  organizations.  In  this  context,  science  is  undeniably 
applied,  solution  oriented  and  focused  to  achieve  strategic  objectives  specified  in  local,  national, 
or  international  research  agendas.  Since  societal,  market,  and  environmental  conditions  are 
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constantly  changing,  the  search  for  and  discovery  of  new  knowledge  is  an  ongoing  process  to 
prepare  mankind  for  future  challenges.  Therefore,  as  two  sides  of  the  same  coin,  basic  and 
applied  science  cannot  exist  without  each  other.  Indeed,  successful  attempts  already  have  been 
undertaken  in  aviation  and  other  domains  to  establish  stronger  connections  between  what  is 
needed  by  the  society  and  what  can  be  provided  by  science  in  terms  of  established  knowledge 
and  solutions. 

We  will  first  take  a  closer  look  at  the  differences  and  communalities  between  the  applied 
and  basic  approaches.  Advantages  and  disadvantages  can  be  found  in  both  types  of  approaches 
along  the  following  dimensions. 

a)  Explanation  —  Solution.  By  identification  of  relations  between  different  sets  of  factual 
information,  basic  science  reduces  the  complexity  within  a  real-world  domain  and 
generates  theories  and  models  that  provide  explanations  for  the  outcomes  of  the  human- 
machine  system  under  investigation.  Applied  research  strives  for  procedural  knowledge, 
which  can  be  utilized  directly  and  provides  solutions  to  operational  needs. 

b)  Controlled  Conditions  —  Natural  Conditions.  Basic  research  is  usually  conducted  within 
a  laboratory  environment  with  tight  control  of  the  experimental  conditions  that  affect  the 
observations.  Since  applied  research  is  oriented  towards  applicable  solutions,  much  of  it 
takes  place  within  the  natural  context  in  which  the  findings  will  be  utilized. 

c)  Scientific  Rigor  —  Customized.  The  awareness  and  control  of  as  many  potential  error 
sources  as  possible  is  one  of  the  main  quality  criteria  of  basic  science.  Applied  science 
has  to  adapt  its  methods  and  instruments  in  accordance  to  the  contextual  conditions 
within  the  field  of  application.  The  dynamics  of  these  conditions  follow  the  natural  flow 
and  cannot  be  fully  predicted. 
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d)  Internal  validity  —  External  validity.  Because  the  experimental  conditions  can  be 
controlled  more  easily  in  laboratory  environments,  results  from  basic  research  often  are 
accorded  a  higher  degree  of  internal  validity.  This  means  more  conclusive  explanations 
can  be  formulated  and  findings  are  more  likely  to  be  replicable  under  equivalent 
conditions.  However,  since  certain  potentially  influential  factors  in  the  natural  conditions 
are  sometimes  intentionally  disengaged  in  the  laboratory  in  order  to  achieve  better 
experimental  control,  the  transferability  of  findings  to  the  world  outside  of  the  laboratory 
is  often  limited.  Experimentally  demonstrated  phenomena  often  cannot  be  observed  in  the 
natural  environment. 

e)  Frequent  Publications  —  Infrequent  Publications.  Basic  research  is  primarily  conducted 
in  universities  and  research  organizations,  where  the  holy  mantra  is  “publish  or  perish.  ” 
In  contrast,  much  of  the  results  of  applied  research  are  not  published.  Although  many 
scientific  journals  intend  to  publish  more  applied  research  findings  in  order  to  attract  a 
wider  audience  from  industry,  the  pragmatism  of  applied  research  is  often  criticized  as 
not  meeting  general  scientific  standards.  In  addition,  applied  research  results  can  affect 
commercial  interests  of  an  organization.  Therefore,  the  interest  to  publish  is  sometimes 
overruled  by  the  need  of  an  organization  to  protect  its  propriety  knowledge,  practice,  and 
technologies. 

f)  Reduced  risk  —  Elevated  risk.  The  level  of  risk  involved  in  the  execution  of  basic  and 
applied  research  differs  significantly.  With  many  factors  under  control  and  a  lower 
complexity  of  possible  interactions,  the  outcome  of  basic  research  is  more  readily 
foreseeable.  Applied  research  often  carries  an  elevated  level  of  risk  because  of  the  higher 
probability  of  unexpected  side-effects  of  an  intervention  or  potential  interactions  with 
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uncontrolled  contextual  influences.  At  the  same  time,  results  from  basic  research  may 
carry  the  risk  of  not  being  generalizable  beyond  the  highly  controlled  laboratory 
environment. 

g)  Proactive  —  Reactive.  Applied  research  is  solution-oriented.  This  means  that  the 

researcher  usually  is  tasked  by  a  customer  or  a  sponsoring  body  to  provide  solutions  for 
an  existing  problem.  The  sponsor  typically  has  full  control  of  the  research  question.  The 
time-horizon  for  a  return  of  the  investment  is  rather  short  because  the  application  of 
research  findings  often  can  start  without  delay,  dependent  only  on  political  decisions. 
Results  of  basic  research  are  usually  more  abstract  and  general.  Applicability  of  the 
findings  is  not  always  immediately  apparent.  General  explanations,  principles,  models 
and  even  identified  problems  are  the  expected  outcomes  from  basic  research.  Because  of 
the  generality,  such  findings  could  also  be  valuable  for  addressing  new  issues  emerging 
in  the  future.  Funding  for  basic  research  is  in  most  cases  provided  from  public  sources 
such  as  scientific  foundations.  The  sponsor  has  less  or  only  indirect  influence  on  how 
exactly  the  funding  is  spent. 

It  is  however  important  to  note  that  these  distinctions  between  basic  and  applied  research 
are  neither  dichotomous  nor  clear  cut.  In  fact,  Stokes  (1997)  argued  that  research  at  the 
intersection  of  the  two  approaches  where  insight  and  usefulness  can  be  of  equal  value  may  be 
particularly  fruitful.  Applied  and  basic  research  each  has  its  merits  and  contributes  to  the 
development  of  the  other.  We  provide  further  evidence  of  the  fact  below  and  advocate  leveraging 
the  strengths  of  each. 

Garnering  the  Fortes  of  Applied  and  Basie  Research 
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In  an  attempt  to  objectively  evaluate  the  relative  merits  of  applied  and  basic  research, 
Adams  (1972)  examined  the  results  of  two  projects.  One  was  Project  Hindsight  that  was 
conducted  by  the  United  States  Department  of  Defense  (Isenson,  1967;  Sherwin  &  Isenson, 

1966).  In  this  project,  20  weapon  systems  were  examined,  among  them  were  defense  systems 
such  as  the  Minuteman  ballistic  missiles,  the  Mark  46  antisubmarine  torpedo,  and  the  Starlight 
Scope  for  passive  night  vision.  The  procedure  was  to  work  backward  from  an  important 
innovation  in  the  system  and  to  ask  what  R&D  events  were  responsible  for  it.  These  R&D  events 
were  traced  back  20  years  to  1945.  Over  700  events  were  found,  91%  of  which  could  be 
classified  as  applied  research.  In  98%  of  these  cases,  the  investigator  was  motivated  by  his 
awareness  that  a  problem  existed,  not  by  pure  scientific  curiosity  or  the  pursuing  of  knowledge. 
Also,  67%  of  the  research  events  occurred  before  the  specific  system  that  it  was  applied  to  was 
begun,  with  a  median  time  of  nine  years  between  occurrence  of  a  research  event  and  its  use  in  a 
weapon  system.  In  other  words,  impactful  research  is  not  short-term  research  that  is  done  on  the 
system  itself,  but  is  often  relatively  long-term  work  that  occurs  well  ahead  of  immediate  need. 
Further,  the  R&D  events  were  found  to  have  influenced  not  just  a  specific  weapon  system  but 
were  applied  to  a  multitude  of  systems. 

The  second  project  was  called  TRACES  (Technology  in  Retrospect  and  Critical  Events  in 
Science,  Loellbach,  1968,  1969).  The  project  examined  long-term  and  basic  research  events  as 
well  as  short-term  applied  ones  without  any  time  restriction.  Critical  R&D  events  associated  with 
five  socially  important  products:  magnetic  ferrites,  the  videotape  recorder,  the  oral  contraceptive 
pill,  the  electron  microscope,  and  matrix  isolation  were  identified.  Over  300  R&D  events  were 
found  and  they  were  classified  into  three  categories:  basic  research,  applied  research,  and 
development  and  application.  As  was  found  in  the  Hindsight  project,  most  of  the  applied  R&D 
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events  occurred  20  years  prior  to  product  innovation.  But  the  basic  research  contributions  came 
even  more  years  before  product  innovation,  peaking  20-30  years  before  the  innovation  and 
mainly  preceding  applied  research.  Moreover,  basic  research  was  far  more  influential  than 
applied  research.  Basic  research,  applied  research,  and  development  and  applications  were  found 
to  be  responsible  for  70%,  20%,  and  10%  of  the  significant  events  respectively. 

The  two  projects  that  Adams  described  convincingly  demonstrated  the  value  and  impact 
of  relatively  long-term  research  and  thereby  the  need  for  researchers  to  continue  to  build  basic 
knowledge  with  rigor.  But  the  time  lag  between  discovery  and  application  could  also  be 
significant.  Stokes,  Adams,  and  many  others  recognized  the  relationship  between  basic  research 
and  applications  need  not  be  serendipitous  (see  also  Helton  &  Kemp,  2011).  Allowing  practical 
needs  to  inform  the  basic  areas  that  need  more  intensive  research  would  be  expected  to  reduce 
the  lag. 

For  example,  Gopher  and  Kimchi  (1989)  identified  three  broad  human  factors  topics  that 
could  benefit  from  better  understanding  of  the  underlying  psychological  principles:  visual 
displays,  mental  workload,  and  training  of  complex  skills.  With  regard  to  the  topic  of  displays, 
recent  technological  developments  have  afforded  practically  infinite  display  format  possibilities. 
Gopher  and  Kimchi  proposed  that  one  principle  that  should  be  used  to  guide  display  designs  is 
the  principle  of  representation.  This  is  because  the  most  effective  display  format  would  probably 
be  the  one  that  is  most  compatible  with  how  information  is  represented  by  the  human  operator. 
Therefore  answers  to  basic  questions  on  how  information  is  represented  are  likely  to  benefit 
many  systems  and  not  just  a  specific  display  of  a  specific  system.  With  regard  to  the  topic  of 
workload,  because  increasing  computerized  automation  has  been  permeating  ever  more  human- 
machine  systems;  there  is  a  growing  need  to  be  able  to  know  how  the  system  is  performing  and 
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to  be  able  to  predict  how  the  system  might  perform.  Measures  of  mental  workload  have  been 
introduced  to  augment  or  complement  vanishing  observable  manual  responses  or  complex 
performance  that  evades  simple  quantification.  Gopher  and  Kimchi  proposed  that  an 
understanding  of  the  variables  that  underlie  changes  in  mental  workload  would  be  tremendously 
useful.  With  regard  to  training,  although  automation  has  certainly  increased  system  capability,  it 
also  introduces  complexity.  This  is  made  plain  from  the  abundance  of  testimonials  from  our 
practitioner  colleagues  in  Chapter  2.  Gopher  and  Kimchi  proposed  that  in  order  to  provide 
effective  and  expedient  training,  we  need  to  have  a  strong  basic  understanding  of  how  learning 
takes  place.  Instead  of  reinventing  the  wheel  for  each  singular  training  occasion,  research  on 
basic  principles  of  skill  developments  will  need  to  continue. 

Importantly,  Gopher  and  Kimchi  argued  that  sound  principles  should  be  applicable  across 
a  wide  range  of  human-machine  systems.  For  example,  a  sound  training  principle  should  be  just 
as  effective  training  a  fighter  pilot,  a  RPA  pilot,  or  an  air  traffic  controller.  Further,  sound 
principles  should  transcend  technologies  of  the  day.  The  existence  of  unique  requirements  in 
each  training  situation  notwithstanding,  validated  training  principles  such  as  the  importance  of 
feedback  and  the  incorporation  of  appropriately  challenging  elements  should  apply  whether  one 
is  learning  how  to  fly  a  plane  with  no  engines,  propellers,  or  jet  engines.  Although  the  years 
since  1989  have  seen  great  advances  in  many  technologies  such  as  sensors  (both  environmental 
and  physiological),  displays  (such  as  3D  and  virtual  displays),  and  especially  computer  hardware 
and  software  (which  enable  unprecedented  levels  of  automation),  the  principles  that  Gopher  and 
Kimchi  (1989)  referenced  are  no  less  relevant  today. 

Role  of  our  professional  organizations.  Adams  (1972)  further  suggested  that 
professional  organizations  could  play  a  more  active  role.  For  aviation  psychology  this  would 
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include  such  organizations  as  the  Applied  Experimental  and  Engineering  Psychology  Division  of 
the  American  Psychological  Association,  the  Human  Factors  and  Ergonomics,  Society  (HFES), 
the  Association  for  Aviation  Psychology  (AAP),  and  the  European  Association  for  Aviation 
Psychology  (EAAP).  All  of  these  organizations  could  be  more  proactive  about  identifying  the 
most  profitable  topics  for  basic  research  that  would  be  likely  to  support  the  most  effective 
transition  to  applied  needs.  Individual  researchers  would,  of  course,  still  plan  their  individual 
research  projects,  but  Adams  suggested  that  a  collective  vision  of  research  priorities  could  help 
direct  efforts  to  topics  that  have  particular  current  relevance.  And  many  such  topics  could  be 
gleaned  from  the  challenges  laid  out  by  our  practitioner  colleagues  in  Chapter  2.  Their  insights 
should  serve  as  fertile  ground  for  identifying  knowledge  gaps  in  our  human  factors  and  aviation 
psychology  database. 

But  how  can  findings  from  highly  controlled  basic  research  be  generalizable  to  the 
real-world?  As  Projects  Hindsight  and  TRACES  have  shown,  generalizations  are  not  only 
possible,  they  are  not  an  anomaly  in  science.  Anderson,  Lindsay,  and  Bushman  (1999) 
considered  the  truism  that  laboratory  research  must  have  low  external  validity  and  field  studies 
must  have  low  validity  and  consequently  little  hope  of  bridging  the  two.  They  used  meta-analytic 
techniques  to  examine  the  consistency  of  the  effects  of  the  same  conceptual  independent 
variables  on  the  same  conceptual  dependent  variables  between  laboratory  and  field  settings 
across  several  domains  in  social  psychology  (e.g.,  weapons  and  aggression,  gender  and 
leadership  style,  age  and  job -training  mastery).  Thirty-eight  pairs  of  laboratory  and  field  effects 
were  found  based  on  a  literature  search  of  the  major  psychological  journals.  Across  domains,  the 
correlation  of  the  effect  size  for  laboratory  and  field  studies  was  .73,  a  correlation  considered  to 
be  large  by  convention  (Cohen,  1988).  The  respectable  correspondence  between  laboratory  and 
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field  findings  showed  that  the  laboratory  findings  examined  must  have  some  external  validity 
and  the  field  findings  must  have  at  least  some  internal  validity.  Anderson  et  al.  argued  that 
laboratory  research  is  by  no  means  inherently  internally  valid  and  field  studies  not.  Scientifically 
unsound  studies  can  be  conducted  in  the  field  as  well  as  in  the  laboratory.  That  is,  neither 
internal  nor  external  validity  is  defined  by  where  the  study  is  conducted  but  by  the  method  with 
which  conclusions  are  drawn. 

However,  as  Chapanis  (1988)  and  many  others  have  cautioned,  generalizations  are  not 
guaranteed  nor  should  they  be  assumed.  But,  there  are  a  number  of  standard  procedures  of  the 
scientific  method  such  as  having  representative  subjects,  providing  sufficient  training,  and  using 
appropriate  measures  that  would  help  improve  the  probability  of  generalizability.  Chapanis  also 
reminded  us  that  one  approach  to  support  generalization  over  a  wide  range  of  situations  is  to 
purposely  design  heterogeneity  into  the  studies.  That  is,  the  same  relationship  should  be  tested 
over  different  subjects  (e.g.,  subjects  with  different  levels  of  experience),  tasks,  response 
measures  (e.g.,  decision  time  and  decision  quality),  and  environmental  conditions  (e.g.,  whether 
the  human  operator  interacts  with  other  humans  or  intelligent  agents). 

Beyond  designing  heterogeneity  in  individual  studies,  Gopher  and  Sanders  (1984) 
advocated  the  back-to-back  strategy  for  the  overall  research  program.  While  the  initial  validation 
of  a  relationship  between  certain  conceptual  variables  and  dependent  measures  would  mostly  be 
done  under  tightly  control  conditions  that  typically  use  simple  tasks,  efforts  need  to  be  made  to 
continue  to  test  the  relationship  with  higher  degree  of  complexity  that  increasingly  approximate 
that  in  the  target  environment  to  which  generalization  is  to  be  made.  Along  the  way,  results  with 
simple  and  more  complex  tasks  are  compared.  This  is  not  only  to  check  the  limits  of  the 
generality  of  the  relationship  being  tested.  This  also  affords  an  opportunity  to  reveal  inaccuracies 
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of  the  theoretical  relationship  and  possibly  ideas  for  a  better  representation,  thereby  contributing 
to  the  existing  knowledge  base. 

Last,  Wickens  and  McCarley  (Chapter  5,  this  volume)  discussed  the  potential  drawbacks 
of  the  conventional  overreliance  on  null-hypothesis  significance  testing  for  inferring  practical 
significance.  The  issue  has  to  do  with  the  overemphasis  on  trying  to  avoid  the  error  of  incorrectly 
rejecting  a  null  hypothesis  or  the  error  of  detecting  an  effect  that  is  not  really  present.  The  risk 
with  this  overemphasis  is  an  increased  probability  of  failure  to  recognizing  an  effect  that  is  in 
fact  present.  While  this  is  a  concern  for  both  basic  and  applied  research,  Wickens  and  McCarly 
argue  that  this  may  be  especially  problematic  in  the  applied  domains  such  as  those  involved  with 
aviation  safety.  For  example,  failure  to  appreciate  a  true  difference  between  two  training 
methods  because  the  difference  did  not  reach  statistical  significance  at  the  conventional  level  of 
p  <  .05,  could  lead  to  the  failure  of  adopting  a  truly  superior  training  method.  While  Wickens  and 
McCarley  are  not  advocating  abandoning  null-hypothesis  significance  testing  altogether, 
researchers  and  practitioners  are  urged  to  also  consider  additional  approaches,  which  they 
described  in  Chapter  5  (this  volume). 

Promoting  Communications  between  Researchers  and  Practitioners 

It  is  abundantly  clear  from  our  practitioner  colleagues  (Chapter  2)  that  there  is  much 
willingness,  even  desire,  to  work  with  researchers  in  a  number  of  capacities.  The  need  to  work 
together  is  equally  clear  to  researchers.  But,  as  discussed  in  Chapter  2,  there  are  considerable 
challenges  to  be  overcome  to  enable  better  communications.  Below  are  a  few  avenues  for 
facilitating  communications  between  practitioners  and  researchers. 

Conferences  and  publications  representing  cross-section  of  practitioners  and 
researchers.  Scientific  journal  publications  have  long  been  the  staple  means  for  researchers  to 
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communicate  with  each  other.  Although  basic  and  applied  research  tended  to  be  published  in 
separate  journals  historically,  that  is  changing.  There  are  now  a  growing  number  of  publications 
with  the  professed  aim  of  bridging  basic  and  applied  research.  They  include  the  Journal  of 
Experimental  Psychology:  Applied ,  Theoretical  Issues  in  Ergonomics  Sciences,  Human  Factors , 
Ergonomics ,  The  International  Journal  of  Aviation  Psychology ,  and  Aviation  Psychology  and 
Applied  Human  Factors ,  just  to  name  a  few.  Periodically,  special  issues  where  the  entire  issue  is 
devoted  to  a  contemporary  topic  receiving  intense  attention  are  put  forth  in  these  journals.  For 
example,  The  International  Journal  of  Aviation  Psychology  has  published  special  issues  on  pilot 
selection  (1996(2),  2014(1/2)),  instructor  training  (2002(3)),  aviation  maintenance  human  factors 
(2008(1)),  synthetic  vision  (2009(1/2))  and  others.  These  special  issues  serve  as  a  particularly 
excellent  forum  for  researchers,  airlines,  manufacturers,  regulators,  and  service  providers  to  all 
examine  and  discuss  the  “real-world”  requirements  together  with  scientifically-proven  solutions. 

Although  many  of  these  journals  are  still  primarily  written  by  and  for  researchers,  they  all 
have  the  requirement  that  the  authors  make  plain  the  relevance  of  the  theoretical  issues  to 
applications.  Some  of  them  also  encourage  practitioners  to  submit  papers  not  only  to  bring 
operational  issues  to  the  attention  of  researchers  but  again,  to  provide  a  common  forum  to  engage 
both  the  researcher  and  practitioner  communities. 

Another  category  of  publications  are  articles  primarily  written  by  researchers  for 
practitioners  to  communicate  new  findings  in  a  language  accessible  to  them.  One  example  is  the 
Ergonomics  in  Design,  published  by  the  Human  Factors  and  Ergonomics  Society.  Certainly, 
having  more  publication  avenues  that  have  the  expressed  aims  of  communicating  with 
practitioners  could  incentivize  researchers  to  work  closely  with  practitioners  in  order  to  produce 
documents  that  are  written  in  a  language  relatable  to  them. 


DISTRIBUTION  STATEMENT  A.  Approved  for  public  release:  distribution  is  unlimited. 


88ABW  Cleared  07/11/2016;  88ABW-2016-3355. 


16 


A  fourth  category  of  publications  are  reports  authored  primarily  by  practitioners.  The 
Aviation  Safety  Reporting  System  (ASRS,  http://asrs.arc.nasa.gov/overview/summary.html) 
accepts  voluntarily  submitted  aviation  safety  incident/situation  reports  from  pilots,  controllers, 
and  others.  The  ASRS  acts  on  these  reports  and  identifies  system  deficiencies,  and  issues  alerting 
messages  to  persons  in  a  position  to  correct  them.  The  authors  are  not  aware  of  the  existence  of  a 
similar  “Hotline”  system  that  would  include  not  just  reports  of  incidents  but  reports  that  identify 
human  factors  deficiencies  or  inefficiencies  much  earlier,  prior  to  the  occurrence  of  incidents.  In 
the  present  highly  networked  world,  such  a  reporting  system  might  provide  a  more  direct 
pathway  for  connecting  practitioners  and  researchers  early  on  in  the  problem  solving  process. 

Even  closer  communication  between  practitioners,  researchers  and  industry 
representatives  is  facilitated  at  conferences  and  workshops  such  as  the  annual  meetings  of  the 
Human  Factors  and  Ergonomics  Society,  the  thematic  lectures  of  the  Royal  Aeronautical 
Society,  and  the  biennial  International  Symposium  of  Aviation  Psychology  and  European 
Association  on  Aviation  Conference. 

Of  note  is  that  in  2014,  the  European  Commission  has  launched  the  OPTICS  project. 
OPTICS  stands  for  “Observation  Platform  for  Technology  and  Institutional  Consolidation  of 
Research  in  Safety.”  It  served  as  a  platform  for  screening  ongoing  safety-related  research  and 
innovation  activities  in  Europe.  OPTICS  organizes  workshops  and  dissemination  events  once  or 
twice  a  year.  These  workshops  offer  the  opportunity  to  engage  with  policy  and  decision  makers 
as  well  as  leading  aviation  safety  researchers  to  confirm  promising  research  avenues  and  adjust 
the  ongoing  and  future  safety  research  agendas.  Further  activities  are  initiated  to  compile  a  living 
repository,  which  traces  existing  and  ongoing  research  and  innovation  activities  with  relevance 
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to  aviation  safety.  This  will  help  to  strengthen  the  accessibility  of  already  generated  knowledge 
and  solutions  and  to  benchmark  it  against  agreed  strategic  goals  and  upcoming  industry  needs. 

Technological  gatekeeper.  Another  approach  for  bringing  the  research  to  the 
practitioners  and  the  practical  challenges  to  the  researchers  is  to  develop  the  role  of  a 
technological  gatekeeper.  Adams  (1972)  suggested  that  this  gatekeeper  would  read  more  of  the 
professional  engineering  and  scientific  journals  than  the  average  practitioner  or  administrator  and 
serve  as  a  translator  of  basic  science.  Also,  the  gatekeeper  would  maintain  a  wide  range  of 
relationships  with  scientists  and  technologists  outside  of  his  organization.  While  there  is  likely  to 
be  someone  in  each  organization  assuming  such  a  role  already,  Adams  (1972)  suggested  such  a 
role  might  be  formalized  and  rewarded  in  order  to  maximize  information  transfer. 

Organizational  climate  and  institutional  support.  Institutional  support  for  facilitating 
the  activities  described  above  is  indispensable.  Many  of  these  activities  like  the  support  for 
publications,  publication  subscriptions,  and  conference  and  workshop  participations,  will 
undoubtedly  incur  costs  but  so  do  ineffectual  designs,  failed  training,  and  unsafe  operations. 
There  are  additional  essential  institutional  supports  that  might  entail  a  paradigm  shift  in  the 
thinking  at  the  operational  sites  as  well  as  in  the  academic  institutions.  For  example  there  needs 
to  be  mechanisms  and  reward  structures  in  the  workplace  for  practitioners  to  be  able  to 
participate  in  research  collaboration  much  more  fully  than  is  currently  typical  (see  Chapter  2). 
Similarly,  academic  researchers  would  need  to  be  able  to  take  certain  risks  in  endeavoring  less 
tightly  controlled  work  that  do  not  necessarily  produce  data  that  are  acceptable  only  in 
theoretical  journals.  That  is,  not  only  would  communications  between  researchers  and 
practitioners  need  to  be  improved,  government,  industry,  and  academic  administrators  very  much 
need  to  be  in  the  loop  as  well.  (See  Figure  2.2  in  Chapter  2). 
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Research  funding  scheme.  An  important  lesson  from  Projects  Hindsight  and  TRACES 
is  that  longer-term  research  in  the  end  could  have  a  much  higher  payoff.  Consequently,  an 
overemphasis  on  funding  only  work  that  seeks  a  quick  but  possibly  only  a  one-time  application 
is  unlikely  to  be  the  winning  strategy.  This  is  especially  important  in  aviation  where  the  time 
needed  to  develop  and  deploy  new  aircraft  or  air  traffic  control  systems  can  require  many  years. 
These  lessons  have  made  some  inroads  into  many  of  the  major  funding  agencies  in  the  United 
States.  At  the  same  time,  major  federal  funding  agencies  that  have  traditionally  funded  primarily 
basic  research  such  as  the  National  Science  Foundation  and  the  National  Institutes  of  Health, 
have  for  some  time  now,  required  grant  proposals  to  include  explicit  statements  of  the  broader 
impact  of  the  proposed  research.  The  European  strategic  planning  effort  for  the  commercial 
travel  system  described  below  is  another  good  example  that  seems  to  have  heeded  this  lesson 
well. 

At  the  Paris  Air  Show  in  June  2001  the  Advisory  Council  for  Aeronautics  Research  in 
Europe  (ACARE)  was  instituted  with  over  40  organizations  across  Europe  and  representatives  of 
the  European  Parliament.  ACARE  was  tasked  to  find  consensus  on  how  aviation  could  better 
serve  society’s  needs  in  the  future.  The  result  was  the  “European  Aeronautics:  A  Vision  for 
2020”  report  (European  Commission,  Group  of  Personalities,  2001).  Since  we  are  presently 
approaching  the  year  2020,  the  vision  was  revised  and  published  as  “Flightpath  2050:  Europe’s 
vision  for  aviation”  in  March  2011  (European  Commission,  High  Level  Group  on  Aviation 
Research,  201 1).  Most  of  Europe’s  national  and  international  funding  schemes  for  research  in 
aviation  (e.g.  Framework  Programs,  Horizon  2020,  SESAR,  CleanSky)  are  based  on  objectives  - 
basic  as  well  as  applied  -  as  defined  in  these  documents. 
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An  important  element  of  ACARE’s  strategy  is  to  establish  a  network  for  strategic 
research  in  aviation  for  the  involved  stakeholders  (including  industry,  research  establishments, 
academia,  regulators  etc.)  and  to  facilitate  stakeholder  co-operation  in  Europe  and 
internationally.  For  example,  a  number  of  specialized  instruments  in  the  Horizon  2020  and 
CleanSky  programs  are  currently  used  to  push  innovation  closer  to  the  market  and  to  stimulate 
the  dialogue  between  academics,  producers  and  end  users.  Thereby,  a  fresh  contract  between 
government  and  science  is  made,  which  will  make  the  case  for  continued  societal  investment  in 
realistic  terms  of  the  problem  solving  capacity  of  science. 

Although  much  of  the  European  2020  and  2050  visions  are  directed  at  non-aviation 
psychology  issues  such  as  cleaner  aircraft  engines  and  recyclable  aircraft,  very  ambitious  goals 
for  aircraft  automation  were  identified.  For  example,  Flightpath  2050  envisioned  a  future  where, 
“Automation  has  changed  the  role  of  both  the  pilot  and  the  air  traffic  controller.  Their  roles  are 
now  as  strategic  managers  and  hands-off  supervisors,  only  intervening  when  necessary.” 
(European  Commission,  High  Level  Group  on  Aviation  Research,  2011,  p.  9). 

The  means  for  achieving  those  goals,  or  indeed  even  proving  their  viability,  will  be  a  tremendous 
challenge  for  aviation  psychology  research  and  must  include  careful  collaboration  between 
researchers  investigating  the  issues  and  practitioners  that  will  ultimately  need  to  use  the  resulting 
systems. 

Forward-Looking,  Future-Oriented  Research 

Innovative  science  does  not  only  depend  on  available  research  facilities  and  financial 
resources.  It  is  the  breed  of  human  curiosity,  bright  minds,  team  spirit,  enthusiasm  and  creativity. 
These  factors  are  often  somewhat  fluctuant  and  can  hardly  be  scheduled  by  duty-rosters  or 
stringent  roadmaps.  Ad-hoc  solutions  to  current  problems  are  rarely  ingenious.  Therefore, 
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science  policy  makers  and  science  managers  have  to  consider  that  today’s  investment  into 
organizational  climate,  individual  promotion,  and  campaigning  for  young  talents  will  serve  our 
society’s  interests  of  tomorrow.  Successful  science  management  will  have  to  find  the  right 
balance  between  controlling  the  expenditure  of  taxpayers'  money  and  maintaining  motivation 
and  creativity-inspiring  tolerances  for  research  within  the  program-oriented  funding  schemes. 
This  is  not  to  say  that  we  should  neglect  current  market  opportunities.  It  means  that  basic 
research  even  without  a  focus  on  immediate  utilitarian  thinking  will  serve  tomorrow’s  needs  if  it 
explicitly  addresses  identified  or  likely  trends  into  future  problems.  Following  the  philosophy  of 
Antoine  de  Saint-Exupery:  "Your  task  is  not  to  foresee  the  future,  but  to  enable  it". 
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